Hierarchical models that address measurement error are needed to evaluate the correlation between treatment effect and control group event rate.

OBJECTIVE: To apply a hierarchical model (HM) that addresses measurement error in regression of the treatment effect on the control group event rate (CR). We compare HM to weighted linear regression (WLR) which is subject to measurement error and mathematical coupling. STUDY DESIGN AND SETTING: We reviewed published hierarchical models that address measurement error and implemented a Bayesian version in open-source code to facilitate adoption by meta-analysts. We compared WLR and HM across a very large convenience sample of meta-analyses published in the Cochrane Database of Systematic Reviews. RESULTS: We applied both approaches (WLR and a HM that addresses measurement error) to 3,193 meta-analyses that included 33,071 studies (average 10.28 studies per meta-analysis). A statistically significant slope suggesting an association between the treatment effect and CR was demonstrated with both approaches in 568 (17.19%) meta-analyses, with neither approach in 2,036 (63.77%) meta-analyses, only with WLS in 229 (7.17%) and only with HM in 360 (11.28%) meta-analyses. The majority of slopes were negative (WLR 85%, HM 83%). In the majority of cases, HM had wider confidence intervals (72.53%) and slopes farther from the null (64.77%). CONCLUSION: Approximately 28% of meta-analyses demonstrate a significant association between the treatment effect and CR when HM is used to address measurement error. User-friendly open-source code is provided to meta-analysts interested in exploring this association.

Influence of lack of blinding on the estimation of medication-related harms: a retrospective cohort study of randomized controlled trials.

BACKGROUND: Empirical evidence suggests that lack of blinding may be associated with biased estimates of treatment benefit in randomized controlled trials, but the influence on medication-related harms is not well-recognized. We aimed to investigate the association between blinding and clinical trial estimates of medication-related harms. METHODS: We searched PubMed from January 1, 2015, till January 1, 2020, for systematic reviews with meta-analyses of medication-related harms. Eligible meta-analyses must have contained trials both with and without blinding. Potential covariates that may confound effect estimates were addressed by restricting trials within the comparison or by hierarchical analysis of harmonized groups of meta-analyses (therefore harmonizing drug type, control, dosage, and registration status) across eligible meta-analyses. The weighted hierarchical linear regression was then used to estimate the differences in harm estimates (odds ratio, OR) between trials that lacked blinding and those that were blinded. The results were reported as the ratio of OR (ROR) with its 95% confidence interval (CI). RESULTS: We identified 629 meta-analyses of harms with 10,069 trials. We estimated a weighted average ROR of 0.68 (95% CI: 0.53 to 0.88, P < 0.01) among 82 trials in 20 meta-analyses where blinding of participants was lacking. With regard to lack of blinding of healthcare providers or outcomes assessors, the RORs were 0.68 (95% CI: 0.53 to 0.87, P < 0.01 from 81 trials in 22 meta-analyses) and 1.00 (95% CI: 0.94 to 1.07, P = 0.94 from 858 trials among 155 meta-analyses) respectively. Sensitivity analyses indicate that these findings are applicable to both objective and subjective outcomes. CONCLUSIONS: Lack of blinding of participants and health care providers in randomized controlled trials may underestimate medication-related harms. Adequate blinding in randomized trials, when feasible, may help safeguard against potential bias in estimating the effects of harms.

A Bayesian nonparametric meta-analysis model for estimating the reference interval.

A reference interval represents the normative range for measurements from a healthy population. It plays an important role in laboratory testing, as well as in differentiating healthy from diseased patients. The reference interval based on a single study might not be applicable to a broader population. Meta-analysis can provide a more generalizable reference interval based on the combined population by synthesizing results from multiple studies. However, the assumptions of normally distributed underlying study-specific means and equal within-study variances, which are commonly used in existing methods, are strong and may not hold in practice. We propose a Bayesian nonparametric model with more flexible assumptions to extend random effects meta-analysis for estimating reference intervals. We illustrate through simulation studies and two real data examples the performance of our proposed approach when the assumptions of normally distributed study means and equal within-study variances do not hold.

A matching-adjusted indirect comparison of the efficacy of elranatamab versus teclistamab in patients with triple-class exposed/refractory multiple myeloma.

For patients with triple-class exposed/refractory multiple myeloma (TCE/RMM), where effective treatments options are limited, B-cell maturation antigen and CD3-directed bispecific antibodies offer a promising new approach. Teclistamab gained conditional approval in Europe and accelerated Food and Drug Administration (FDA) approval based on the MajesTEC-1 trial (NCT03145181). Elranatamab, approved by the FDA demonstrated its safety and efficacy in the MagnetisMM-3 trial (NCT04649359). Given the absence of head-to-head trials, an unanchored matching-adjusted indirect comparison (MAIC) was conducted to assess their relative efficacy. Key baseline characteristics were adjusted to be comparable between the two trials. In the MAIC, elranatamab demonstrated significantly better objective response rate and progression-free survival (PFS) than teclistamab, and numerically better complete response, duration of response, and overall survival (OS). These results suggest that elranatamab is an efficacious option for treating patients with TCE/R MM.

A comprehensive review and shiny application on the matching-adjusted indirect comparison.

Population-adjusted indirect comparison (PAIC) is an increasingly used technique for estimating the comparative effectiveness of different treatments for the health technology assessments when head-to-head trials are unavailable. Three commonly used PAIC methods include matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multilevel network meta-regression (ML-NMR). MAIC enables researchers to achieve balanced covariate distribution across two independent trials when individual participant data are only available in one trial. In this article, we provide a comprehensive review of the MAIC methods, including their theoretical derivation, implicit assumptions, and connection to calibration estimation in survey sampling. We discuss the nuances between anchored and unanchored MAIC, as well as their required assumptions. Furthermore, we implement various MAIC methods in a user-friendly R Shiny application Shiny-MAIC. To our knowledge, it is the first Shiny application that implements various MAIC methods. The Shiny-MAIC application offers choice between anchored or unanchored MAIC, choice among different types of covariates and outcomes, and two variance estimators including bootstrap and robust standard errors. An example with simulated data is provided to demonstrate the utility of the Shiny-MAIC application, enabling a user-friendly approach conducting MAIC for healthcare decision-making. The Shiny-MAIC is freely available through the link: https://ziren.shinyapps.io/Shiny_MAIC/.

Association Between Maternal Depression and Lower Urinary Tract Symptoms in Their Primary School-Age Daughters: A Birth Cohort Study.

PURPOSE: Although maternal depression is associated with adverse outcomes in women and children, its relationship with lower urinary tract symptoms (LUTS) in offspring is less well-characterized. We examined the association between prenatal and postpartum maternal depression and LUTS in primary school-age daughters. DESIGN: Observational cohort study. SUBJECTS AND SETTING: The sample comprised 7148 mother-daughter dyads from the Avon Longitudinal Study of Parents and Children. METHOD: Mothers completed questionnaires about depressive symptoms at 18 and 32 weeks' gestation and 21 months postpartum and their children's LUTS (urinary urgency, nocturia, and daytime and nighttime wetting) at 6, 7, and 9 years of age. Multivariable logistic regression models were used to estimate the association between maternal depression and LUTS in daughters. RESULTS: Compared to daughters of mothers without depression, those born to mothers with prenatal and postpartum depression had higher odds of LUTS, including urinary urgency (adjusted odds ratio [aOR] range = 1.99-2.50) and nocturia (aOR range = 1.67-1.97) at 6, 7, and 9 years of age. Additionally, daughters born to mothers with prenatal and postpartum depression had higher odds of daytime wetting (aOR range = 1.81-1.99) and nighttime wetting (aOR range = 1.63-1.95) at 6 and 7 years of age. Less consistent associations were observed for depression limited to the prenatal or postpartum periods only. CONCLUSIONS: Exposure to maternal depression in the prenatal and postpartum periods was associated with an increased likelihood of LUTS in daughters. This association may be an important opportunity for childhood LUTS prevention. Prevention strategies should reflect an understanding of potential biological and environmental mechanisms through which maternal depression may influence childhood LUTS.

Double-Negative Results Matter: A Reevaluation of Sensitivities for Detecting SARS-CoV-2 Infection Using Saliva Versus Nasopharyngeal Swabs.

In a recent systematic review, Bastos et al. (Ann Intern Med. 2021;174(4):501-510) compared the sensitivities of saliva sampling and nasopharyngeal swabs in the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection by assuming a composite reference standard defined as positive if either test is positive and negative if both tests are negative (double negative). Even under a perfect specificity assumption, this approach ignores the double-negative results and risks overestimating the sensitivities due to residual misclassification. In this article, we first illustrate the impact of double-negative results in the estimation of the sensitivities in a single study, and then propose a 2-step latent class meta-analysis method for reevaluating both sensitivities using the same published data set as that used in Bastos et al. by properly including the observed double-negative results. We also conduct extensive simulation studies to compare the performance of the proposed method with Bastos et al.'s method for varied levels of prevalence and between-study heterogeneity. The results demonstrate that the sensitivities are overestimated noticeably using Bastos et al.'s method, and the proposed method provides a more accurate evaluation with nearly no bias and close-to-nominal coverage probability. In conclusion, double-negative results can significantly impact the estimated sensitivities when a gold standard is absent, and thus they should be properly incorporated.

A matching-adjusted indirect comparison of the efficacy of elranatamab versus physician's choice of treatment in patients with triple-class exposed/refractory multiple myeloma.

INTRODUCTION: For patients with triple-class exposed/refractory multiple myeloma (TCE/R MM), prognosis is poor and effective treatment options are limited. Elranatamab is a novel B-cell maturation antigen (BCMA)- and CD3-directed bispecific antibody which was approved by the US Food and Drug Administration in August 2023 and demonstrated safety and efficacy in patients with TCE/R MM in the phase 2, single-arm MagnetisMM-3 trial (NCT04649359). To compare the effectiveness of elranatamab vs physician's choice of treatment (PCT) in the absence of head-to-head comparative data, a matching-adjusted indirect comparison (MAIC) was conducted. METHODS: Individual patient data from MagnetisMM-3 (Cohort A [BCMA-naïve] N = 123, 14.7 months of follow-up) were reweighted to match published summary data from two real-world studies of PCT in patients with TCE/R MM (LocoMMotion and MAMMOTH) using a propensity score-type logistic regression. Unanchored MAIC analyses were conducted according to National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU) 18 guidance. RESULTS: Compared with PCT in LocoMMotion, elranatamab was associated with a significantly higher objective response rate (ORR rate difference: 37.52; 95% CI 26.20-48.83; odds ratio: 4.85; 95% CI 2.85-8.23) and complete or stringent complete response rate (≥CR rate difference: 42.29; 95% CI 31.84-52.74; odds ratio: 184.01; 95% CI 24.66-1372.86), longer progression-free survival (PFS HR 0.32; 95% CI 0.20-0.49), and overall survival (OS HR 0.62; 95% CI 0.40-0.94). Compared with PCT in MAMMOTH, elranatamab was associated with significantly higher ORR (rate difference: 28.14; 95% CI 16.77-39.52; odds ratio: 3.24; 95% CI 1.98-5.32) and ≥ CR (rate difference: 26.22; 95% CI 16.40-36.05; odds ratio: 5.48; 95% CI 2.88-10.44), as well as longer PFS (HR 0.25; 95% CI 0.17-0.37) and OS (HR 0.49; 95% CI 0.33-0.71). Sensitivity analysis results were consistent with the base case. CONCLUSION: In the MAIC, elranatamab was consistently associated with improved rates and depth of response and significantly longer PFS and OS versus PCT in LocoMMotion and MAMMOTH.

Causally interpretable meta-analysis: Clearly defined causal effects and two case studies.

Meta-analysis is commonly used to combine results from multiple clinical trials, but traditional meta-analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently-introduced causally interpretable meta-analysis methods and apply their treatment effect estimators to two individual-participant data sets. These estimators transport estimated treatment effects from studies in the meta-analysis to a specified target population using the individuals' potentially effect-modifying covariates. We consider different regression and weighting methods within this approach and compare the results to traditional aggregated-data meta-analysis methods. In our applications, certain versions of the causally interpretable methods performed somewhat better than the traditional methods, but the latter generally did well. The causally interpretable methods offer the most promise when covariates modify treatment effects and our results suggest that traditional methods work well when there is little effect heterogeneity. The causally interpretable approach gives meta-analysis an appealing theoretical framework by relating an estimator directly to a specific population and lays a solid foundation for future developments.

Network meta analysis to predict the efficacy of an approved treatment in a new indication.

Drug repurposing refers to the process of discovering new therapeutic uses for existing medicines. Compared to traditional drug discovery, drug repurposing is attractive for its speed, cost, and reduced risk of failure. However, existing approaches for drug repurposing involve complex, computationally-intensive analytical methods that are not widely used in practice. Instead, repurposing decisions are often based on subjective judgments from limited empirical evidence. In this article, we develop a novel Bayesian network meta-analysis (NMA) framework that can predict the efficacy of an approved treatment in a new indication and thereby identify candidate treatments for repurposing. We obtain predictions using two main steps: first, we use standard NMA modeling to estimate average relative effects from a network comprised of treatments studied in both indications in addition to one treatment studied in only one indication. Then, we model the correlation between relative effects using various strategies that differ in how they model treatments across indications and within the same drug class. We evaluate the predictive performance of each model using a simulation study and find that the model minimizing root mean squared error of the posterior median for the candidate treatment depends on the amount of available data, the level of correlation between indications, and whether treatment effects differ, on average, by drug class. We conclude by discussing an illustrative example in psoriasis and psoriatic arthritis and find that the candidate treatment has a high probability of success in a future trial.

Regulatory agilities impacting review timelines for Pfizer/BioNTech's BNT162b2 mRNA COVID-19 vaccine: a retrospective study.

The appropriate use of regulatory agilities has the potential to accelerate regulatory review, utilize resources more efficiently and deliver medicines and vaccines more rapidly, all without compromising quality, safety and efficacy. This was clearly demonstrated during the COVID-19 pandemic where regulators and industry rapidly adapted to ensure continued supply of existing critical medicines and review and approve new innovative medicines. In this retrospective study, we analyze the impact of regulatory agilities on the review and approval of Pfizer/BioNTech's BNT162b2 mRNA COVID-19 Vaccine globally using regulatory approval data from 73 country/regional approvals. We report on the critical role of reliance and provide evidence that demonstrates reliance approaches and certain regulatory agilities reduced review times for the COVID-19 vaccine. These findings support the case for more widespread implementation of regulatory agilities and demonstrate the important role of such approaches to improve public health outcomes.

Bayesian hierarchical models incorporating study-level covariates for multivariate meta-analysis of diagnostic tests without a gold standard with application to COVID-19.

When evaluating a diagnostic test, it is common that a gold standard may not be available. One example is the diagnosis of SARS-CoV-2 infection using saliva sampling or nasopharyngeal swabs. Without a gold standard, a pragmatic approach is to postulate a "reference standard," defined as positive if either test is positive, or negative if both are negative. However, this pragmatic approach may overestimate sensitivities because subjects infected with SARS-CoV-2 may still have double-negative test results even when both tests exhibit perfect specificity. To address this limitation, we propose a Bayesian hierarchical model for simultaneously estimating sensitivity, specificity, and disease prevalence in the absence of a gold standard. The proposed model allows adjusting for study-level covariates. We evaluate the model performance using an example based on a recently published meta-analysis on the diagnosis of SARS-CoV-2 infection and extensive simulations. Compared with the pragmatic reference standard approach, we demonstrate that the proposed Bayesian method provides a more accurate evaluation of prevalence, specificity, and sensitivity in a meta-analytic framework.

The association of sensitivity and specificity with disease prevalence: analysis of 6909 studies of diagnostic test accuracy.

BACKGROUND: Sensitivity and specificity are characteristics of a diagnostic test and are not expected to change as the prevalence of the target condition changes. We sought to evaluate the association between prevalence and changes in sensitivity and specificity. METHODS: We retrieved data from meta-analyses of diagnostic test accuracy published in the Cochrane Database of Systematic Reviews (2003-2020). We used mixed-effects random-intercept linear regression models to evaluate the association between prevalence and logit-transformed sensitivity and specificity. The model evaluated all meta-analyses as nested within each systematic review. RESULTS: We analyzed 6909 diagnostic test accuracy studies from 552 meta-analyses that were included in 92 systematic reviews. For sensitivity, compared with the lowest quartile of prevalence, the second, third and fourth quartiles were associated with significantly higher odds of identifying a true positive case (odds ratio [OR] 1.17, 95% confidence interval [CI] 1.09-1.26; OR 1.32, 95% CI 1.23-1.41; OR 1.47, 95% CI 1.37-1.58; respectively). For specificity, compared with the lowest quartile of prevalence, the second, third and fourth quartiles were associated with significantly lower odds of identifying a true negative case (OR 0.74, 95% CI 0.69-0.80; OR 0.65, 95% CI 0.60-0.70; OR 0.47, 95% CI 0.44-0.51; respectively). Pooled regression coefficients from bivariate models conducted within each meta-analysis showed that prevalence was positively associated with sensitivity and negatively associated with specificity. Findings were consistent across subgroups. INTERPRETATION: In this large sample of diagnostic studies, higher prevalence was associated with higher estimated sensitivity and lower estimated specificity. Clinicians should consider the implications of disease prevalence and spectrum when interpreting the results from studies of diagnostic test accuracy.

Borrowing Concurrent Information from Non-Concurrent Control to Enhance Statistical Efficiency in Platform Trials.

A platform trial is a trial involving an innovative adaptive design with a single master protocol to efficiently evaluate multiple interventions. It offers flexible features such as dropping interventions for futility and adding new interventions to be evaluated during the course of a trial. Although there is a consensus that platform trials can identify beneficial interventions with fewer patients, less time, and a higher probability of success than traditional trials, there remains debate on certain issues, one of which is whether (and how) the non-concurrent control (NCC) (i.e., patients in the control group recruited prior to the new interventions) can be combined with the current control (CC) in the analysis, especially if there is a change of standard of care during the trial. METHODS: In this paper, considering time-to-event endpoints under the proportional hazard model assumption, we introduce a new concept of NCC concurrent observation time (NCC COT), and propose to borrow NCC COT through left truncation. This assumes that the NCC COT and CC are comparable. If the protocol does not prohibit NCC patients to change the standard of care while on study, NCC COT and CC likely will share the same standard of care. A simulated example is provided to demonstrate the approach. RESULTS: Using exponential distributions, the simulated example assumes that NCC COT and CC have the same hazard, and the treatment group has a lower hazard. The estimated HR comparing treatment to the pooled control group is 0.744 (95% CI 0.575, 0.962), whereas the comparison to the CC group alone is 0.755 (95% CI 0.566, 1.008), with corresponding p-values of 0.024 versus 0.057, respectively. This suggests that borrowing NCC COT can improve statistical efficiency when the exchangeability assumption holds. CONCLUSION: This article proposes an innovative approach of borrowing NCC COT to enhance statistical inference in platform trials under appropriate scenarios.

Proposed triggers for retiring a living systematic review.

Living systematic reviews (LSRs) are systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs are critical for decision-making in topics where the evidence continues to evolve. It is not feasible to continue to update LSRs indefinitely; however, guidance on when to retire LSRs from the living mode is not clear. We propose triggers for making such a decision. The first trigger is to retire LSRs when the evidence becomes conclusive for the outcomes that are required for decision-making. Conclusiveness of evidence is best determined based on the GRADE certainty of evidence construct, which is more comprehensive than solely relying on statistical considerations. The second trigger to retire LSRs is when the question becomes less pertinent for decision-making as determined by relevant stakeholders, including people affected by the problem, healthcare professionals, policymakers and researchers. LSRs can also be retired from a living mode when new studies are not anticipated to be published on the topic and when resources become unavailable to continue updating. We describe examples of retired LSRs and apply the proposed approach using one LSR about adjuvant tyrosine kinase inhibitors in high-risk renal cell carcinoma that we retired from a living mode and published its last update.

An improved Bayesian approach to estimating the reference interval from a meta-analysis: Directly monitoring the marginal quantiles and characterizing their uncertainty.

Reference intervals, or reference ranges, aid medical decision-making by containing a pre-specified proportion (e.g., 95%) of the measurements in a representative healthy population. We recently proposed three approaches for estimating a reference interval from a meta-analysis based on a random effects model: a frequentist approach, a Bayesian posterior predictive interval, and an empirical approach. Because the Bayesian posterior predictive interval becomes wider to incorporate estimation uncertainty, it may systematically contain greater than 95% of measurements when the number of studies is small or the between study heterogeneity is large. The frequentist and empirical approaches also captured a median of less than 95% of measurements in this setting, and 95% confidence or credible intervals for the reference interval limits were not developed. In this update, we describe how one can instead use Bayesian methods to summarize the appropriate quantiles (e.g., 2.5th and 97.5th) of the marginal distribution of individuals across studies and construct a credible interval describing the estimation uncertainty in the lower and upper limits of the reference interval. We demonstrate through simulations that this method performs well in capturing 95% of values from the marginal distribution and maintains a median coverage of near 95% of the marginal distribution even when the number of studies is small, or the between-study heterogeneity is large. We also compare the results of this method to those obtained from the three previously proposed methods in the original case study of the meta-analysis of frontal subjective postural vertical measurements.

RIMeta: An R shiny tool for estimating the reference interval from a meta-analysis.

A reference interval, or an interval in which a prespecified proportion of measurements from a healthy population are expected to fall, is used to determine whether a person's measurement is typical of a healthy individual. For a specific biomarker, multiple published studies may provide data collected from healthy participants. A reference interval estimated by combining the data across these studies is typically more generalizable than a reference interval based on a single study. Methods for estimating reference intervals from random effects meta-analysis and fixed-effects meta-analysis have been recently proposed and implemented using R software. We present an R Shiny tool, RIMeta, implementing these methods, which allows users not proficient in R to estimate a reference interval from a meta-analysis using aggregate data (mean, standard deviation, and sample size) from each study. RIMeta (https://cers.shinyapps.io/RIMeta/) provides users a convenient way to estimate a reference interval from a meta-analysis and to generate the reference interval plot to visualize the results. The use of this web-based R Shiny tool does not require the installation of R or any background knowledge of programming. We explain all functions of the R Shiny tool and illustrate how to use it with a real data example.